Hiding Quiet Solutions in Random Constraint Satisfaction Problems 
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We study constraint satisfaction problems on the so-called planted random ensemble. We show that for 
a certain class of problems, e.g. graph coloring, many of the properties of the usual random ensemble are 
quantitatively identical in the planted random ensemble. We study the structural phase transitions, and the 
easy/hard/easy pattern in the average computational complexity. We also discuss the finite temperature phase 
diagram, finding a close connection with the liquid/glass/solid phenomenology. 
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Constraint satisfaction problems (CSPs) stand at the root of 
the theory of computational complexity [IJ and arise in com- 
puter science, physics, engineering and many other fields of 
science. Consider a set of N discrete variables and a set of M 
Boolean constraints; the problem consists in finding a configu- 
ration of variables that satisfies all the constraints or in proving 
that no such configuration exists. Algorithmical approaches to 
intrinsically hard NP-complete CSPs [ 1 ] are one of the biggest 
challenges in today's science. Ensembles of random CSPs, 
where the constraints are chosen uniformly at random from 
some prescribed distribution, are being used to understand the 
average computational complexity ||2 S- Techniques from 
statistical physics of glassy systems have allowed us to shed 
new light on the problem and on the origin of the average al- 
gorithmical hardness 

A major point in evaluating the performance of new algo- 
rithms for hard CSPs is to be able to generate difficult in- 
stances that are guaranteed to be satisfiable. Planting is the 
most standard way to do so: one first chooses a configura- 
tion of variables and then considers only constraints which 
are compatible with this planted configuration. Many plant- 
ing protocols have been introduced iQHUst], however, the un- 
derstanding of when and why they provide a difficult instance 
compared 
iii. 

tion changes the properties of the ensemble. It is moreover 
often anticipated that the planted solution is easier to find than 
a random one, as has been indeed proven for high density of 
constraints flO, 12ll . Hard instances with a known solution 



is still very poor compared to what is known for the purely 
random ensemble Ij, S la] . This is because planting a solu- 



are also appealing to cryptographic application as they provide 
good one-way functions. Planted instances may also result 
from applications where only constraints compatible with an 
initial state of the system are added. 

In this Letter we show that for a specific, yet large, class 
of CSPs, one can easily generate planted instances by hid- 
ing a quiet solution that does not have influence on most of 
the characteristics of the ensemble. The canonical example 
of a CSP where a solution can be planted in the quiet way is 
the graph g-coloring problem on which we shall illustrate our 
findings about the phase diagram and the average algorithmi- 
cal hardness. The class of problems that allow a simple quiet 
hiding will be discussed towards the end of the Letter. 



Hiding without changing — The graph coloring problem 
consists in deciding if the N vertexes of a graph can be col- 
ored using only q colors in such a way that every two adjacent 
vertices have different colors. The control parameter is the 
average degree of variables c, and we consider the thermody- 
namical limit ^ oo. In statistical physics, this problem 
coiTesponds to a Potts antiferromagnet 11131 

The way to plant a quiet solution in the graph coloring prob- 
lem is actually the most natural one: One assigns a random 
color with equal probability to each of the A^ vertices, and then 
constructs the graph by randomly throwing links between ver- 
tices of different colors. Using the cavity method [15] we de- 
scribe the phase diagram and the structure of solutions in this 
planted ensemble. In the large A^ limit, the degree distribution 
in the planted graphs is Poissonian with mean c, and thus they 
are locally tree-like just as the standard random Erdos-Renyi 
graphs. Following the cavity approach ifTil ITsll the Belief- 
Propagation (BP) equations can be written. Denote ipl'^^ the 
probability that the site i takes color s in absence of the site j: 
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(1) 
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where Z*^^ is a normalization ensuring that ^'^j^ V's^"' — 1- 
The entropy (the logarithm of the number of proper colorings) 
is computed from the fixed point of eq. ([T]) as 



s=l jedi 

q 



s=l 



(2) 



The entropy per site s = S/N can thus be computed if the 
distribution P(V') over the graph is known. Assuming the ab- 
sence of long range congelations, recursive equations on this 
distribution can be written and solved via the population dy- 
namics technique lITsll . In the planted ensemble one needs to 
distinguish between the sites that were planted with different 
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colors, we thus consider q different distributions: 



k\ iq-lY 

k=Q ' si...,Sfc 



where Si are all the possible colors but s, s is taking values 
1; . . . , g, and function /(■) was defined in eq. ([T]). The fixed 
point of (O may depend on the initial conditions. One might 
initialize (i/;) randomly, or in the planted solution itself, i.e., 
all the elements in Ps{ip) are vectors fully oriented in the di- 
rection of the color s. The dependence on initial conditions is 
a generic sign for the appearance of different Gibbs states. 

Before discussing further properties of the planted ensem- 
ble let us review briefly those of the purely random ensemble. 
The space of solutions in the coloring of random graphs un- 
dergoes several transitions as average degree c is increased 
||^[l3l: For low enough degree, c < Cd, almost all solutions 
(proper colorings of the graph) belong to a single Gibbs state 
and the problem can be studied using the BP approach. For 
c > Cd, the space of solutions shatters into exponentially 
many different clusters/states, each corresponding to a dif- 
ferent Gibbs state. In this case, a technique called one-step 
replica symmetry breaking (IRSB) |4j,il5i] is used to describe 
the phase space. To focus on clusters of a given size we intro- 
duce the Parisi IRSB parameter m, clusters are then weighted 
by their size to the power of m lfl4l [l6l. For c < Cc a typ- 
ical solution belongs to a cluster corresponding to the value 
771 = 1. For c > Cc, although exponentially many clusters ex- 
ist, a random solution will with high probability belong to one 
of the few largest clusters, corresponding to < m < 1, while 
the TO = 1 clusters do not exist anymore; this is called the con- 
densed phase 1 6]. Finally for c> Cs the last cluster disappears 
and no solutions exist anymore. On top of this geometrical 
behavior in the space of solutions, a remarkable phenomenon 
appears within the clusters themselves. In some of them a 
finite fraction of variables are allowed only one color (a phe- 
nomenon c?l\\ freezing) [ 14l[l7t]. To the best of our knowledge, 
no existing algorithm is able to find solutions in the frozen 
clusters in polynomial time ||5l[l4, 23], and since in a region 
near to the colorability threshold all clusters are frozen this 
provides a bound on the algorithmically hard phase. 

Coming back to the planted ensemble, notice that eq. (O is 
nothing else but the IRSB equation for the coloring of purely 
random graphs at to = 1 (compare e.g. with eq. (C4) in liljll . 
or with the equations for the reconstruction on trees lIlSll ). It 
is known from 1 18] that if ^ is initialized in the planted con- 
figuration then in the fixed point the distribution Ps is biased 
towards color s above the reconstruction threshold, i.e., for 
average degree c > Cd The value Cd is then a spinodal 
point for the existence of a planted Gibbs state containing the 
planted configuration. From the equivalence of eq. ^ with 
the IRSB equation at to = 1 for the purely random ensem- 
ble it also follows that both the planted and the purely ran- 
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FIG. 1 : (color online) Phase diagram on the 5-coloring on the planted 
ensemble. Bottom: Sketches of the clustering. At an average degree 
Cd the space of solutions shatters into exponentially many clusters, 
the planted cluster being one of them. Beyond Cc the planted cluster 
contains more solutions than all the others together. At the last 
non-planted cluster disappears. Top: Total entropy Stot with the sub- 
dominant part (dashed). The equilibrium complexity Eequiubrium 
(logarithm of the number of dominant clusters), the entropy of the 
non-planted clusters and critical degrees are taken from L i 4.1 . 



dom ensembles admit the so-called liquid solution where all 
i/'s = 1/9. A linear stability analysis shows that the liquid so- 
lution is locally stable against small perturbations towards the 
planted solution for c < c/ = (g — 1)^ |24]. Above c; the only 
stable fixed point of (O is strongly biased towards the planted 
configuration and c; is therefore a spinodal point for the liquid 
state. The fact that the liquid solution is stable for c < c; also 
means that the properties of the phase space are not affected 
by the very existence of the planted state. This leads us to 
the important conclusion — which we call quiet planting — 
that in this region the properties of the planted ensemble are 
exactly the same as the properties of the purely random en- 
semble, up to the existence of the planted state 12511 . 

For completeness let us mention that, just as in the purely 
random ensemble fTT], the liquid solution in the planted en- 
semble decomposes further into IRSB states fore > Cd- Prop- 
erties of these states can be obtained by solving the IRSB 
equations. In the planted ensemble, the IRSB equations have 
only one nontrivial solution independent from the planted 
configuration ll26[l and it is identical to the IRSB solution in 
the purely random ensemble. Since the liquid state is identi- 
cal in the two ensembles, it is not so surprising than the same 
conclusion applies to the glassy phase. 

Phase diagram of planted coloring — We now describe 
the phase diagram of the planted ensemble (Fig.[T]). Up to the 
average degree Cd almost all solutions belong to one single 
large cluster/state of entropy sbp ~ log(7+(c/2) log(l — l/g). 
Above Cd the space of solutions splits into exponentially many 
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clusters, as in the purely random ensemble. As the planted 
cluster/state is described by the solution of dS]) it has all the 
properties of the m = 1 clusters from the purely random en- 
semble. In particular, for Cd < c < Cc the planted cluster is 
one of the exponentially many equilibrium clusters and thus 
for c < Cc the purely random and planted ensembles of ran- 
dom graphs are asymptotically equivalent. Interestingly, this 
equivalence has been rigorously proven in ll22ll . however only 



up to an average degree Cq < 02711 . 



For c > Cc all the non-planted m — 1 clusters disappear 
and the size of the planted cluster becomes larger than the to- 
tal size of the remaining ones. A first order transition happens 
and the planted state dominates the total number of solution. 
The entropy is given by the fixed point of (O initialized in the 
planted solution plugged into (|2]i. Another transition appears 
at Cs (the colorability threshold in the purely random ensem- 
ble) beyond which all clusters disappears except the planted 
one. The values of Cd, Cc and Cg given q are identical to those 
in the purely random ensemble, and are listed in [14]. 

The properties of the planted cluster can be studied nu- 
merically on a single graph (as was done for satisfiability in 
Jgtl). We checked on many instances that the BP equations ^ 
initialized in the planted configuration converge to the liquid 
fixed point for c < Cd, and to the planted one for c > Cd, while 
when initialized randomly they converge to the planted fixed 
point only for c > c;. We have also considered the appearance 
of frozen variables discussed in lIlTIl : Fig. [2] shows the agree- 
ment between the fraction of frozen variables obtained by ap- 
plying the whitening procedure 1 19] to the planted configura- 
tion on a given graph with the theoretical prediction lIlTll . 

Easy/Hard/Easy pattern — If one does not discover the 
planted cluster, the planted and the purely random graphs are 
indistinguishable and we thus expect that in such a case they 
have comparable algorithmic difficulty. This is indeed what 
we observed in experiments with several solvers. Fig.|2]shows 
results of the Walk-COL algorithm lll411 on both the planted 
and purely random graphs. No difference is visible, thus (un- 
less the planted cluster intervenes) the easy/hard pattern ob- 
served in the colorable phase c < is the same in both the 
ensembles. It has been empirically argued this transition is 
related to the freezing of clusters [141]. 

On the other hand, for very large degree c ^ Cs it is known 
that even simple message passing algorithms find a solution 
near to the planted one [12]: therefore a second hard/easy 
transition must exist. This is due to the aforementioned linear 
instability at c;: Since for c > q BP ([T]i converges sponta- 
neously towards the planted fixed point (as shown in Fig. |2]i 
it is easy to find solutions from the planted cluster above ci 
(e.g. BP decimation algorithm finds solutions in linear time). 
For c < C(, however, without prior knowledge of the planted 
configuration BP converges to the uniform liquid fixed point. 
Applying the state of art algorithms (BP decimation, BP rein- 
forcement, Walk-COL, simulated annealing, etc.) to planted 
instances for < c < c; we were indeed not able to find 
solutions in polynomial time. This suggests that the hard/easy 
algorithmic transition in the planted ensemble arises exactly at 
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FIG. 2: (color online) Fraction of variables frozen in their planted 
colors in 5-coloring of a A'^ = 10'' graph. Data obtained from the 
BP fixed point when initialized randomly (full triangles) and in the 
planted configuration (also called the whitening [19], empty circles), 
compared to the theoretical predictions 1 14, 17] (full and dashed 
lines). For c > ci = Id BP converges spontaneously to the planted 
fixed point. For c < 14.04 [17] there are no frozen variables in the 
planted cluster Inset: Fraction of monochromatic edges versus the 
number of sweeps of the Walk-COL algorithm 1 14] in 5-coloring of 
a purely random and a planted graph, TV = 10"". Quiet planting does 
not seem to affect the computational hardness in the region c < Cs. 



c;. Note at this point that since Cd — Q for q — 3, the planted 
3-coloring is algorithmically easy for all degrees. 

Phase diagram at finite temperature — It is finally of in- 
terest to consider the properties of the problem at finite tem- 
perature T, using a unit energy cost for every monochromatic 
edge. The BP and the IRSB equations can be easily extended 



to this situation as e.g. in 11141 uOT . Note, however, that at 
T > the equations for the planted ensemble do not corre- 
spond anymore to the IRSB equations at m = 1, making the 
finite temperature problem richer. In fact, the system behaves 
just as a usual mean field glass problem, with its liquid/glass 
transition, where the planted state acts as a solid-like (or crys- 
tal) phase. This solid/planted phase exists bellow an upper 
spinodal temperature Ti (that starts at Cd, see Fig.[3]l, and the 
liquid solution ijjs = 1/9 becomes unstable towards this solid 
state as it encounters a spinodal point at 



T2 = -1/log 



1 



(4) 



which starts at c; at T = (see Fig. O. As usual in first or- 
der phase transitions, the free energy of the liquid and solid 
state have to be compared to draw the equilibrium phase tran- 
sition line, starting at Cc alT = 0. As in the purely random 
ensemble the liquid state undergoes a dynamical and Kauz- 
mann glass transition (Fig. |3]l. The 3-coloring is particular: 
the two spinodals coincide, making the equilibrium transition 
of a second order A similar phase diagram as in Fig. |3]was 
found in l2li] for the ferromagnetic p-spin model, in fact that 
model is just a particular case of our quiet planting setting. 



4 




10 12 '^d '^c 14 16 18 20 

Average degree 

FIG. 3: (color online) Finite temperature phase diagram of the 
planted 5-coloring. At high temperature the liquid is the unique 
Gibbs state. Below the higher spinodal line Ti a planted (solid) 
Gibbs state appears, and becomes thermodynamically dominant at 
the liquid/solid transition. The super-cooled liquid state is locally sta- 
ble until the lower spinodal line T2. The liquid undergoes the usual 
dynamical and Kauzmann (ideal) glass transitions (data from |20]). 
The thick line depicts the algorithmically hard region, the left bound- 
ary being the freezing transition |T4]. Inset: Energy versus temper- 
ature in Monte-Carlo annealing with rate ST = 10~^, 10"^, 10"* 
and 10^'' per sweep for A'' = 5. 10'' at c = 20 > c;. Above q a slow 
enough annealing undergoes a transition towards the planted state. 



In 112 111 , however, the liquid state is always stable and finding 
the ground state is polynomial since the problem reduces to a 
set of linear equations. 

The inset of Fig. [3] shows the behavior of Monte-Carlo 
annealing for c > c; to illustrate the liquid/glass/solid phe- 
nomenology: Upon lowering the temperature, the liquid can 
be super-cooled, the solid phase avoided and a glass transition 
observed. But with slow enough annealing the system transits 
to the solid (planted) state at temperature T2. In this case a 
simple simulated annealing is able to find the ground state. If 
the system is initialized in the planted solution and the temper- 
ature is increased, the solid will melt to the liquid state at tem- 
perature Ti . For connectivities c < c; the absence of the liquid 
spinodal line and the mean field nature of the model (barriers 
between states are extensive) makes algorithms based on lo- 
cal dynamics unable to find the planted cluster. This gives a 
physical interpretation behind the hard/easy transition at c;. 

Conclusion — We have discussed the graph coloring 
problem on the planted ensemble and showed that quantita- 
tive results, explicitly checked via numerical experiments, can 
be readily deduced from what is known in the purely ran- 
dom ensemble. In particular we showed that for c < Cc the 
planted ensemble is asymptotically equivalent to the purely 
random one. Many works have established the easiness of the 
planted ensemble at very large ifioi [Tl , 12 1 and very small 
average degree. We bridged the gap and showed that while 
the easy/hard transition in the planted ensemble is similar to 
the one in the usual random ensemble, the hard/easy transi- 
tion coincides with a local instability of the liquid phase at 
ci = (g — 1)^. This leaves a large region of very hard problems 



with a known hidden solution. We also showed how to create 
mean field "glass" models with a solid-like/planted state. 

Let us finish by discussing the (large) class of CSPs where 
the quiet planting is possible and where our phenomenol- 
ogy readily apply. The crucial property that we used when 
stating that the natural planting does not change much of 
the structural properties was the uniformity of the BP fixed 
point in the purely random ensemble (e.g. in coloring t/j — 
{1/q, . . . , 1/q)). Many other CSPs actually share this prop- 
erty, e.g. all the problems without disordered interactions on 
random regular graphs, the hyper-graph bicoloring |23], or the 
balanced locked problems of |5]. These last ones are partic- 
ularly appealing as hard satisfiable benchmarks. The random 
satisfiability problem, however, is a canonical example where 
the fixed point of the BP equation is not uniform and where 
our results do not apply. It would be interesting to generalize 
our approach to plant quiet solutions in such cases. 
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